Motion detecting device and search region variable-shaped motion detector

ABSTRACT

An overall motion detecting device ( 1 ) includes a search region variable-shaped motion detector ( 2 ) and an overall motion information generator ( 3 ). The overall motion information generator ( 3 ) controls a surveillance camera ( 5 ) which is an external image capturing system and supplies control information of the surveillance camera ( 5 ) (such as moving direction, angular speed and focal length) to the search region variable-shaped motion detector ( 2 ) as overall motion information (S 3 ). The search region variable-shaped motion detector ( 2 ) calculates a motion vector (mv) while renewing, by picture, a search region having a search shape determined by the overall motion information (S 3 ) and a region area suitable for its operational capability.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a motion detecting device forgenerating a motion vector, applicable to video encoding standards usingmotion compensation such as MPEG-1, MPEG-2 and MPEG-4.

2. Description of the Background Art

Among conventional motion (vector) detecting devices for determiningsearch ranges is an optimum motion vector search section in a videoencoding device disclosed in Japanese Patent Application Laid-Open No.11-252571 (1999).

This optimum motion vector search section includes a search rangedetermining part and a predicted motion vector peripheral search part.The search range determining part obtains the size of search range basedon an allowable calculation amount given by an external control part.The predicted motion vector peripheral search part determines the shapeof search region based on the distribution of motion vectors alreadyobtained, and the size of search range as the size determined by thesearch range determining part.

In a conventional motion detecting device, the size of search region(search range) varies according to an allowable calculation amount, sothat the search range is determined as having such a size that preventsthe device from sufficiently exerting the processing capability. This isone of factors that disadvantageously prevent effective generation of amotion vector.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a motion detectingdevice capable of effectively generating a motion vector.

According to a first aspect of the invention, the motion detectingdevice for generating a motion vector in image data to be encodedincludes an overall motion information generator and a search regionvariable-shaped motion detector. The overall motion informationgenerator is configured to generate overall motion informationindicative of overall motion of an image specified by the image data.The search region variable-shaped motion detector is configured todetermine a search shape as a determined search shape based on theoverall motion information and to set a search region having an areadetermined by the determined search shape and operational capability ofthe search region variable-shaped motion detector, and to perform acomparison operation in the search region between current image data andreference image data in the image data, thereby generating a motionvector. The determined search shape varies at a predetermined timing asthe overall motion information varies.

The motion vector is obtained while varying the determined search shapeat the predetermined timing as the overall motion information varies.Thus, it is possible to obtain the motion vector of high accuracy witheffective processing.

Further, the search region is set to have a region area based on theoperational capability of the search region variable-shaped motiondetector. Thus, it is always possible to set the search region having aregion area suitable for the operational capability of the search regionvariable-shaped motion detector even when the determined search shapevaries.

According to a second aspect of the invention, the search regionvariable-shaped motion detector for detecting a motion vector in imagedata to be encoded includes a search region shape control part and amotion detecting part. The search region shape control part isconfigured to receive overall motion information indicative of overallmotion of an image and to determine a search shape as a determinedsearch shape based on the overall motion information and to set a searchregion having an area determined by the determined search shape andpredetermined operational capability, and to output a selection signalfor selecting, by a predetermined unit, at least one of current imageand reference image in the image data in accordance with the searchregion. The motion detecting part is configured to perform a comparisonoperation between the reference image and the current image in the imagedata based on the selection signal, thereby detecting the motion vector.The predetermined operational capability includes operational capabilityof the motion detecting part.

The motion vector is obtained while varying the determined search shapeat the predetermined timing as the overall motion information varies.Thus, it is possible to obtain the motion vector of high accuracy witheffective processing.

Further, the search region is set to have a region area based on theoperational capability of the search region variable-shaped motiondetector. Thus, it is always possible to set the search region having aregion area suitable for the operational capability of the search regionvariable-shaped motion detector even when the determined search shapevaries.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of a motiondetecting device according to a first preferred embodiment of thepresent invention;

FIG. 2 is a block diagram illustrating the internal configuration of asearch region variable-shaped motion detector shown in FIG. 1;

FIG. 3 is a first illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region;

FIG. 4 is a second illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region;

FIG. 5 is a third illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region;

FIG. 6 is an illustrative example of search shapes determined by asearch region shape control part shown in FIG. 2;

FIG. 7 is a block diagram illustrating the configuration of an MPEG-2video encoding device according to a second preferred embodiment of theinvention;

FIG. 8 is an illustrative example of the difference between the traveldistances of subjects in accordance with the angle at which asurveillance camera moves and focal lengths;

FIG. 9 is an illustrative example of search shapes determined by asearch region shape control part in a motion detecting device accordingto a third preferred embodiment of the invention;

FIG. 10 illustrates examples of search shapes determined by a searchregion shape control part in a motion detecting device according to afourth preferred embodiment of the invention;

FIG. 11 is a first illustrative example of the search shapes obtained ina motion detecting device according to a fifth preferred embodiment ofthe invention;

FIG. 12 is a second illustrative example of the search shapes obtainedin the motion detecting device according to the fifth preferredembodiment;

FIG. 13 is a first illustrative example of the search shapes obtained ina motion detecting device according to a sixth preferred embodiment ofthe invention;

FIG. 14 is a second illustrative example of the search shapes obtainedin the motion detecting device according to the sixth preferredembodiment;

FIG. 15 is a table showing an example of details on a search regionselected by a search region shape control part based on overall motioninformation in a motion detecting device according to a seventhpreferred embodiment of the invention;

FIG. 16 is an illustrative example of the search shapes determined by asearch region shape control part according to an eighth preferredembodiment of the invention;

FIG. 17 is an illustrative example of macroblocks on an image in a ninthpreferred embodiment of the invention;

FIG. 18 is an illustrative example of a search region for a firstmacroblock shown in FIG. 17;

FIG. 19 is an illustrative example of a search region for a secondmacroblock shown in FIG. 17;

FIG. 20 is an illustrative example of a search region for a thirdmacroblock shown in FIG. 17;

FIG. 21 is a block diagram illustrating the configuration of a motiondetecting device according to a tenth preferred embodiment of theinvention;

FIG. 22 is a flow chart illustrating a process of generating overallmotion information performed by an overall motion information generator;

FIG. 23 is a flow chart illustrating details of step ST1 shown in FIG.22;

FIG. 24 is a flow chart illustrating details of step ST2 shown in FIG.22;

FIG. 25 is a block diagram illustrating the configuration of an MPEG-2video encoding device according to a tenth preferred embodiment of theinvention;

FIG. 26 is a block diagram illustrating the configuration of anotherMPEG-2 video encoding device according to the tenth preferredembodiment;

FIG. 27 is a block diagram illustrating the internal configuration of asearch region variable-shaped motion detecting device according to aneleventh preferred embodiment of the invention;

FIG. 28 is a first illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region;

FIG. 29 is a second illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region;

FIG. 30 is a third illustrative example of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region; and

FIG. 31 is an illustrative example of the feature according to a twelfthpreferred embodiment of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Preferred Embodiment

FIG. 1 is a block diagram illustrating the configuration of a motiondetecting device according to a first preferred embodiment of thepresent invention. As shown, a motion detecting device 1 includes asearch region variable-shaped motion detector 2 and an overall motioninformation generator 3. The generator 3 controls a surveillance camera5 which is an external image capturing system.

The overall motion information generator 3 supplies control informationof the surveillance camera 5 (such as moving direction, angular speedand focal length) to the search region variable-shaped motion detector 2as overall motion information S3.

For instance, the motion of the surveillance camera 5 is generallycontrolled by control information of a microcontroller. Thus, theoverall motion information generator 3 having the function of suchmicrocontroller may supply the search region variable-shaped motiondetector 2 with the control information itself as the overall motioninformation S3. The control information is information indicative ofoverall motion of an image specified by image data captured by thesurveillance camera 5.

FIG. 2 is a block diagram illustrating the internal configuration of thesearch region variable-shaped motion detector 2. As shown, the motiondetector 2 includes a search region shape control part 21 and a motiondetecting part 22. The shape control part 21 determines the shape ofsearch region based on the overall motion information S3 obtained fromthe overall motion information generator 3, then sets a search region ofregion area determined by the above-mentioned determined search shapeand operational capability of the detecting part 22, and outputs acurrent image selection signal S11, a reference image selection signalS12 and the other selection signal S13 for realizing the search region,to the detecting part 22.

Here, the search region represents a region that corresponds to part ofa screen for detecting a motion vector. The region area represents thearea of search region, and the search shape represents the shape ofsearch region. A current image is an image to be encoded, and areference image is a previous or subsequent image with respect to thecurrent image, used for comparison with the current image. Respectivepixel values of the current image and reference image are compared toobtain a motion vector. The motion vector represents a vector indicativeof the extent and direction of motion of part of an image between framesor the like.

Upon receipt of reference image data SW in image data obtained from thesurveillance camera 5 and current image data TMB (usually obtained whenencoding the image data), The motion detecting part 22 performs acomparison operation between reference image data SW and current imagedata TMB in a search region determined by the current image selectionsignal S11 and reference image selection signal S12, thereby outputtinga motion vector mv based on the result of comparison.

FIGS. 3 to 5 are illustrative examples of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region. In these examples, it isassumed that the number of macroblocks including 16-by-16 pixels or thelike that can be selected by each of reference image data SW and currentimage data TMB is four, operational capability of the motion detectingpart 22 is eight cycles and a comparison between reference image data SWand current image data TMB can be performed by macroblock in one cycle.In FIGS. 3 to 5, reference images SW1 to SW4 represent macroblocksadjacent to one another in the vertical direction, and current imagesTMB1 to TMB4 represent macroblocks adjacent to one another in thehorizontal direction. By reducing the space between adjacent ones of thereference images SW1 to SW4 letting the images be partly overlapped, thebasic unit of a search region can be made smaller than a macroblock.Thus, a search region can be realized on a pixel basis at the minimum.

In a first selection example 31 shown in FIG. 3, the reference imagesSW1 and SW2 are each subjected to a comparison operation with the fourcurrent images TMB1 to TMB4, thereby determining a horizontally-orientedrectangular search region 35 having an area of 8 macroblocks.

In a second selection example 32 shown in FIG. 4, the reference imagesSW1 to SW4 are each subjected to a comparison operation with the twocurrent images TMB2 and TMB3, thereby determining a vertically-orientedrectangular search region 36 having an area of 8 macro bocks.

In a third selection example 33 shown in FIG. 5, the reference image SW1is subjected to a comparison operation with the two current images TMB1and TMB2, the reference images SW2 and SW3 with the two current imagesTMB3 and TMB3, and the reference image SW4 with the two current imagesTMB3 and TMB4, thereby determining a search region 37 having an area of8 macroblocks, extending downwardly to the right.

FIG. 6 is an illustrative example of search shapes determined by thesearch region shape control part 21. As shown, any one of the followingthree rectangular search shapes is selected: a horizontally-orientedrectangular search shape 41; a square search shape 42; and avertically-oriented rectangular search shape 43.

Upon receipt of the overall motion information S3, the search regionshape control part 21 selects the search shape 41 when the overallmotion information S3 indicates an inclination in the horizontaldirection, selects the search shape 43 when the information S3 indicatesan inclination in the vertical direction, or selects the search shape 42when the information S3 indicates that there is no inclination in eitherthe horizontal or vertical direction. The above explains onlyillustrative examples of search shapes, and the number of search shapesand how search shapes are selected are not limited as above described.

The timing at which the search region shape control part 21 renews asearch shape shall be on a picture (one screen) basis. For instance,search is conducted with the search shape 41 for five initial screensjudged as including many kinds of motion in the lateral (horizontal)direction, then, with the search shape 42 for next ten screens on whichmotion is gentle, and thereafter, with the search shape 43 for nextthree screens judged as including many kinds of motion in thelongitudinal (vertical) direction. A minimum control unit as the timingof renewing a search shape shall be a picture of image data.

As described, the motion detecting device of the present embodimentcalculates the motion vector mv while renewing, by picture, a searchregion having a search shape determined by the overall motioninformation S3 which is the control information on the motion of thesurveillance camera 5. Thus, it is possible to obtain the motion vectormv of high accuracy with effective processing.

In addition, by setting the region area of search region based on theoperational capability of the motion detecting part 22, a search regionof region area suitable for the operational capability of the motiondetecting part 22 can always be set even if the selected search shapevaries. Conversely saying, the present embodiment can vary the searchshape based on the overall motion information S3 while maintaining theregion area (calculation amount) of search region constant.

In combination with the renewal of search shape according to the presentembodiment, the direction with respect to a reference point of a searchregion may be shifted.

Further, while the present embodiment has described the motion detectingdevice 1 including the search region variable-shaped motion detector 2and overall motion information 3, the motion detector 2 itself has theeffect of obtaining the above-described motion vector mv of highaccuracy with effective processing by receiving the overall motioninformation S3 from outside.

Second Preferred Embodiment

FIG. 7 is a block diagram illustrating the configuration of an MPEG-2video (/audio/system) encoding device 10 according to a second preferredembodiment of the invention. The present embodiment describes, by way ofexample, that a control device (the overall motion information generator3) for the surveillance camera 5 and an image encoding device areprovided separately, and a search region variable-shaped motion detector7 corresponding to the search region variable-shaped motion detector 2of the first preferred embodiment is part of the MPEG-2 video encodingdevice 10.

In this configuration, the overall motion information S3 from theoverall motion information generator 3 is transmitted to the searchregion variable-shaped motion detector 7 through a host I/F 11.

Hereinafter, the internal configuration of the MPEG-2 video encodingdevice 10 will be described. The host I/F 11 transmits the overallmotion information S3 from the overall motion information generator 3onto a (processor) bus 27 and a (host) bus 28.

A DSP part 12 is connected to the bus 28 and performs various kinds ofdigital signal processing. A video I/F 13 is connected to the buses 27to 29 and performs interface processing on a video input SV1 and a videooutput SV0.

An inner loop processing part 14 is connected to the buses 27 to 29 andperforms inner loop processing such as orthogonal transform,quantization, inverse quantization and inverse orthogonal transform, andsupplies a quantized signal to a variable-length encoding part 15.

The variable-length encoding part 15 is connected to the buses 27 to 29and performs variable-length encoding processing based on a signalobtained from the inner loop processing part 14.

The video input SV1, after passing through the video I/F 13, isprocessed in the inner loop processing part 14 and stored in an externalprediction memory through a DRAM I/F 18, whereby the reference data SWis obtained.

The video input SV1, after passing through the video I/F 13, is storedin the prediction memory through DRAM I/F 18, whereby the current imagedata TMB is obtained. Thus, the reference image data SW and currentimage data TMB are captured from the prediction memory into the searchregion variable-shaped motion detector 7 through the DRAM I/F 18.

The search region variable-shaped motion detector 7 performs acomparison operation between reference image data SW and current imagedata TMB through the DRAM I/F 18 in a search region determined based onthe overall motion information S3 obtained through the host I/F 11,thereby calculating a motion vector mv, similarly to the search regionvariable-shaped motion detector 2 of the first preferred embodiment.

A bit stream I/F 16 is connected to the buses 27 to 29 and outputs a bitstream signal BS0 in accordance with a predetermined order oftransmitting encoded data.

An audio I/F 17 is connected to the buses 27 to 29 and performsinterface processing on an audio input SA1.

The DRAM I/F 18 is connected to the buses 27 to 29 and performsinterface processing on the input and output data in the above-mentionedprediction memory.

An overall control part 19 controls the operations of the components 2and 13 to 18 except for the host I/F 11 and DSP part 12.

In such configuration, the search region variable-shaped motion detector7 calculates the motion vector mv based on the overall motioninformation S3 obtained from the overall motion information generator 3externally provided, and the MPEG-2 video encoding device 10 performsencoding based on the motion vector mv. Therefore, the same effects canbe obtained as those in the first preferred embodiment.

The overall motion information S3 may include the moving direction,angular speed, focal length of the surveillance camera 5 and the like.Since the range of motion varies depending on the focal length even atthe same angular speed, information on the focal length is important toobtain a suitable search shape.

FIG. 8 is an illustrative example of the difference between the traveldistances of subjects in accordance with the angle at which thesurveillance camera 5 moves and focal lengths.

As shown, in the case where a subject to be captured is a screen 181having a focal length L1, the screen 181 moves only slightly when thesurveillance camera 5 moves at an angle θ. However, in the case of ascreen 182 having a focal length L2 (>L1), the screen 182 moves a greatdistance away when the camera 5 moves at the same angle θ. In otherwords, as the focal length increases, the extent of motion on a screenat the same angle increases. This shows the importance of information onthe focal length in the overall motion information S3.

Third Preferred Embodiment

FIG. 9 is an illustrative example of search shapes determined by thesearch region shape control part 21 in a motion detecting deviceaccording to a third preferred embodiment of the invention. An overallconfiguration may be that of the first preferred embodiment shown inFIGS. 1 and 2 or that of the second preferred embodiment shown in FIG.7.

As shown in FIG. 9, rhombus is adopted as the basic search shape. Asearch shape 61 is a horizontally-oriented rhombus, a search shape 62 isa rhombus whose sides in the vertical and horizontal directions havealmost the same length, and a search shape 63 is a vertically-orientedrhombus. A search region having any of these search shapes 61 to 63 isrealized, for example, on a pixel basis as a structural unit.

Upon receipt of the overall motion information S3, the search regionshape control part 21 of the present embodiment selects the search shape61 when the overall motion information S3 indicates an inclination inthe horizontal direction, selects the search shape 63 when theinformation S3 indicates an inclination in the vertical direction, orselects the search shape 62 when the information S3 indicates that thereis no inclination in the horizontal or vertical direction. The otheroperations are the same as those in the first preferred embodiment.

Therefore, the motion detecting device of the present embodiment exertsthe same effects as those in the first preferred embodiment. Further,the motion detecting device of the present embodiment determines thebasic search shape as rhombus. Accordingly, in the same region area(calculation amount) of search region, it is possible to determine asearch region having a greater inclination than in the first preferredembodiment in which the basic search shape is rectangle. Thus, themotion vector mv can be calculated more effectively. Further, when asearch region is realized on a pixel basis, it is possible to determinea rhombus search region of high accuracy in shape.

Although the present embodiment has described the three types of searchshapes, they are not limited as such, and any one of arbitrary n types(n>1) may be selected.

Fourth Preferred Embodiment

FIG. 10 illustrates examples of search shapes determined by a searchregion shape control part in a motion detecting device according to afourth preferred embodiment of the invention. The overall configurationmay be that of the first preferred embodiment shown in FIGS. 1 and 2 orthat of the second preferred embodiment shown in FIG. 7.

As shown in FIG. 10, parallelogram (including rectangle) is adopted as abasic search shape. Search shapes 71 and 72 are parallelograms extendingupward to the left, a search shape 73 is a rectangle, and search shapes74 and 75 are parallelograms extending upward to the right. A searchregion having any of these search shapes 71 to 75 is realized, forexample, on a pixel basis.

Upon receipt of the overall motion information S3, the search regionshape control part 21 of the present embodiment selects either of thesearch shapes 71 and 72 when the overall motion information S3 indicatesan inclination in the direction upward to the left or downward to theright, selects either of the search shapes 74 and 75 when theinformation S3 indicates an inclination in the direction upward to theright or downward to the left, or selects the search shape 73 when theinformation S3 indicates that there is no inclination in any of thedirections upward to the left, downward to the right, upward to theright or downward to the left. The other operations are the same asthose in the first preferred embodiment.

Therefore, the motion detecting device of the present embodiment exertsthe same effects as those in the first preferred embodiment. Further,the motion detecting device of the present embodiment determines thebasic search shape as parallelogram, resulting in the effect that themotion vector mv can be calculated more effectively with respect to themotion in a diagonal direction on a screen of an image captured by thesurveillance camera 5. Further, when a search region is realized on apixel basis, it is possible to determine a parallelogrammic searchregion of high accuracy in shape.

Although the present embodiment has described the five types of searchshapes, they are not limited as such, and any one of arbitrary n types(n>1) may be selected.

Fifth Preferred Embodiment

FIGS. 11 and 12 are illustrative examples of search shapes obtained in amotion detecting device according to a fifth preferred embodiment of theinvention. In the present embodiment, a search region of rhombus searchshape according to the third preferred embodiment is realized on amacroblock basis.

As shown in these drawings, a rhombus search region is artificiallyrealized using small square macroblocks MB. A search region 81 shown inFIG. 11 is an example in which a horizontally-oriented rhombus(corresponding to the search shape 61 shown in FIG. 9) is artificiallyrealized by macroblocks MB. A search region 82 shown in FIG. 12 is anexample in which a rhombus whose sides in the vertical and horizontaldirections have almost the same length (corresponding to the searchshape 62 shown in FIG. 9) is artificially realized by macroblocks MB.

In FIGS. 11 and 12, the number of macroblocks MB is 48, respectively.That is, the search regions 81 and 82 can be realized by the sameoperational capability (region area).

Since the basic operation is the same as that in the third preferredembodiment, the motion detecting device of the present embodiment exertsthe same effects as those in the third preferred embodiment. That is,the motion detecting device of the present embodiment can determine asearch region having a greater inclination with the same operationalcapability, similarly to the third preferred embodiment, resulting inthe effect that the motion vector mv can be calculated more effectively.Further, since a search region is realized on a macroblock basis, acomparison control between reference image data SW and current imagedata MTB is advantageously easier than in the case of a search regionformed on a pixel basis.

Although the present embodiment has described the two types of searchshapes, they are not limited as such, and any one of arbitrary n types(n>1) may be selected.

Sixth Preferred Embodiment

FIGS. 13 and 14 are illustrative examples of search shapes obtained in amotion detecting device according to a sixth preferred embodiment of theinvention. In the present embodiment, a parallelogrammic search regionaccording to the fourth preferred embodiment is realized on a macroblockbasis.

As shown in these drawings, a parallelogrammic search region isartificially realized using macroblocks MB. A search region 91 shown inFIG. 13 is an example in which a gently inclined parallelogram extendingupward to the right (corresponding to the search shape 71 shown in FIG.10) is artificially realized by macroblocks MB. A search region 92 shownin FIG. 12 is an example in which a relatively sharply inclinedparallelogram extending upward to the left (corresponding to the searchshape 71 shown in FIG. 10) is artificially realized by macroblocks MB.

In FIGS. 13 and 14, the number of macroblocks MB is 48, respectively.That is, the search regions 91 and 92 can be realized by the sameoperational capability.

Since the basic operation is the same as that in the fourth preferredembodiment, the motion detecting device of the present embodiment exertsthe same effects as those in the fourth preferred embodiment. That is,the motion detecting device of the present embodiment exerts the effectthat the motion vector mv can be calculated more effectively withrespect to the motion in a diagonal direction on a screen of an imagecaptured by the surveillance camera 5. Further, since a search region isrealized on a macroblock basis, a comparison control between referenceimage data SW and current image data MTB is advantageously easier thanin the case of a search region formed on a pixel basis.

Although the present embodiment has described the two types of searchshapes, they are not limited as such, and any one of arbitrary n types(n>1) may be selected.

Seventh Preferred Embodiment

In the first to sixth preferred embodiments, one of various searchshapes of one basic search shape (any of rectangle, rhombus andparallelogram) is selected. In a seventh preferred embodiment of theinvention, one of various types of basic shapes is selected as searchshape.

FIG. 15 is a table showing an example of details on selection of asearch region performed by a search region shape control part based onoverall motion information S3 in a motion detecting device according tothe present embodiment.

As shown, the search shape is determined based on the direction andextent of motion indicated by the overall motion information S3. Forinstance, rhombus is selected when the motion is in thehorizontal/vertical direction with relatively great extent (i.e., whenrectangle is not suitable), and rectangle is selected when the motion isin the horizontal/vertical direction with relatively small extent (i.e.,when rectangle is suitable). Parallelogram is selected when the motionis in a diagonal direction.

Since the basic operation is the same as that in the first preferredembodiment, the motion detecting device of the present embodiment exertsthe same effects as those in the first preferred embodiment. Further,the motion detecting device of the present embodiment changes the basicshape of search region based on the overall motion information S3, sothat there are many variations of basic shapes adaptable to obtainedoverall motion. Thus, it is possible to calculate the motion vector mvof high accuracy with more effective processing. A search region may berealized on a pixel basis or a macroblock basis.

Although the present embodiment has described the three types ofselectable basic shapes, they are not limited as such, and any one ofarbitrary m types (m>1) of basic shapes may be selected.

Eighth Preferred Embodiment

While the search shape is determined maintaining search accuracyconstant in the first preferred embodiment, the search accuracy is oneof factors in determining the search shape in an eighth preferredembodiment of the invention.

FIG. 16 is an illustrative example of search shapes determined by asearch region shape control part 21. As shown, similarly to the firstpreferred embodiment shown in FIG. 6, any search shape is selected fromamong the search shapes 41 to 43 and search shapes 104, 105 both havingan area twice that of any of the search shapes 41 to 43 (i.e., halfsearch accuracy).

The search shapes 104 and 105 are both determined to have an area twicethat of any of the search shapes 41 to 43 and to have search accuracyhalf that of any of the search shapes 41 to 43. For instance, when thesearch accuracy of the search shapes 41 to 43 is set by one pixel, thesearch accuracy of the search shapes 104 and 105 is set by two pixels(by skipping one pixel). Thus, a search region of any of the searchshapes 41 to 43, 104 and 105 can be selected with the same calculationamount.

Since the operations except for the selection of search shapes are thesame as that in the first preferred embodiment, the motion detectingdevice of the present embodiment exerts the same effects as those in thefirst preferred embodiment. Further, the motion detecting device of thepresent embodiment can also change the search accuracy based on theoverall motion information S3. For instance, when there is littleinclination in overall motion, either of the search shapes 104 and 105is selected emphasizing the region area rather than the search accuracy.When there is a significant inclination in overall motion, any of thesearch shapes 41 to 43 can be selected emphasizing the search accuracy.

That is, with the motion detecting device of the present embodiment,there are many variations of search regions adaptable to obtainedoverall motion. Thus, the motion vector mv of high accuracy can becalculated more effective processing.

Although the present embodiment has described selectable basic shapes oftwo types, they are not limited as such, and any basic shape ofarbitrary k types (k>1) may be selected.

Ninth Preferred Embodiment

The first preferred embodiment has described that the renewal timing(predetermined timing) of search region (search shape) is on a picturebasis. In a ninth preferred embodiment of the invention, the searchshape and search region (including the direction with respect to areference point) are determined by slice (a plurality of macroblocks)not by picture.

FIG. 17 is an illustrative example of macroblocks on an image 110. Asshown, a macroblock 111 is in contact with the top end of a screen, amacroblock 112 is present in the center of the screen, and a macroblock113 is in contact with the bottom end of the screen.

FIGS. 18 to 20 are illustrative examples of search regions for themacroblocks 111 to 113, respectively. Here, it is assumed that theoverall motion information S3 outputted from the overall motioninformation generator 3 indicates that the motion in the horizontaldirection is significant and that a search region is set in a registeror the like in the search region shape control part 21. The originalpoint (the intersection of X and Y axes) shown in FIGS. 18 to 20 shallbe a reference point for determining the direction of forming a searchregion.

As shown in FIG. 18, a search region 121 is determined for a slicecomprised of the topmost group of macroblocks including the macroblock111. Since the macroblock 111 is present at the top end of the screenand a vertical vector in the negative direction (upward direction of thescreen) is not necessary, the search region is set only in the positivedirection with respect to the original point.

As shown in FIG. 20, a search region 123 is determined for a slicecomprised of the bottommost group of macroblocks including themacroblock 113. Since the macroblock 113 is present at the bottom end ofthe screen and a vertical vector in the positive direction (downwarddirection of the screen) is not necessary, the search region is set onlyin the negative direction with respect to the original point.

As shown in FIG. 19, for slices including the macroblock 112 other thanthe above slices, a search region 122 is set as usual equally in thepositive and negative directions with respect to the original point.

Since the basic operation is the same as that in the first preferredembodiment, the motion detecting device of the present embodiment exertsthe same effects as those in the first preferred embodiment. Further,the motion detecting device of the present embodiment changes settingrequirements for a search region (including the direction with respectto the original point) by slice. Thus, the motion vector mv can becalculated more effectively.

For instance, with respect to an image in which an upper region withrespect to the center of a viewpoint moves rapidly in the upwarddirection and a lower region moves rapidly in the downward direction,such as an image of a scene seen by a driver of a car moving with highspeed, a search region is set by slice. For instance, a search regionsuch as the search region 123 is set for searching only in the upwarddirection in the above-mentioned upper region and a search region suchas the search region 121 for searching only in the downward direction inthe above-mentioned lower region. Thus, the motion vector mv of highaccuracy can be calculated with more effective processing in the presentembodiment.

Although the present embodiment only describes the basic shape of searchregion as rectangular, other shapes may be adopted.

Tenth Preferred Embodiment

FIG. 21 is a block diagram illustrating the configuration of a motiondetecting device according to a tenth preferred embodiment. As shown, amotion detecting device 130 includes the search region variable-shapedmotion detector 2 and an overall motion information generator 4.

The overall motion information generator 4 generates overall motioninformation S4 based on motion vector related information S2 obtained bythe search region variable-shaped motion detector 2 including all motionvectors mv per picture and their evaluation values (such as the sum ofabsolute values of differences which are factors in determining themotion vectors).

FIG. 22 is a flow chart illustrating a process of generating the overallmotion information S4 performed by the overall motion informationgenerator 4. Referring to the flow chart, overall high-reliabledistribution data is generated in step ST1.

FIG. 23 is a flow chart illustrating details of the step ST1 shown inFIG. 22.

Referring to the flow chart, in step ST11, horizontal vectordistribution processing and vertical vector distribution processing inall motion vectors in one picture are carried out based on horizontaland vertical components of a motion vector obtained by a predeterminedunit (for example, by macroblock MB), thereby obtaining a horizontalvector distribution value VX and a vertical vector distribution value VYas well as obtaining overall vector distribution data D1 indicative ofvector distribution of all motion vectors.

Next, in step ST12, a motion vector whose evaluation value is lower thana threshold value is selected as a high-reliable motion vector fromamong all motion vectors. Only such high-reliable motion vector is atarget of processing in step ST13.

In step ST13, horizontal vector distribution processing and verticalvector distribution processing are carried out based on horizontal andvertical components of the high-reliable motion vector, therebyobtaining a high-reliable horizontal vector dispersion value HVX and ahigh-reliable vertical vector dispersion value HVY as well as obtaininghigh-liable vector distribution data D2 indicative of vectordistribution of all high-reliable motion vectors.

Referring back to FIG. 22, the process shifts to overall motion judgmentin step ST2. FIG. 24 is a flow chart illustrating details of step ST2shown in FIG. 22.

Referring to the flow chart, in step ST21, it is checked whether or notthe number of high-reliable motion vectors is equal to or higher than apredetermined reference value. If yes, the process shifts to step ST23.If no, the process shifts to step ST22.

In step ST22, when the number of high-reliable motion vectors is lowerthan the reference value, the overall motion information S4 is generatedbased on the overall vector distribution data D1 such that a searchregion is limited along an axis where motion vectors are widelydistributed. For instance, when low-reliable overall motion vectors arewidely distributed in the vertical direction in the state that thesearch shape 43 shown in FIG. 6 is selected, the overall motioninformation S4 is generated which estimates a search in the verticaldirection results in bad encoding efficiency and thus indicates thatextension in the vertical direction should be prevented (i.e., extensionin the horizontal direction should be promoted). As a result, the searchregion variable-shaped motion detector 2 changes the search shape 43 tosearch shape 42 based on the overall motion information S4.

In step ST23, when the number of high-reliable motion vectors is equalto or higher than the reference value, the overall motion information S4is generated based on the high-reliable vector distribution data D2 suchthat a search region is limited along an axis where motion vectors aresparsely distributed. For instance, when high-reliable overall motionvectors are widely distributed in the vertical direction in the statethat the search shape 42 shown in FIG. 6 is selected, the overall motioninformation S4 is generated which estimates that there is little motionin the vertical direction and thus indicates that extension in thevertical direction should be prevented. As a result, the search regionvariable-shaped motion detector 2 changes the search shape 42 to searchshape 43 based on the overall motion information S4.

That is, the overall motion information S4 generated in step ST22indicates a motion in a direction where low-reliable overall motionvectors are sparsely distributed, while the information S4 generated instep ST23 indicates a motion in a direction where high-reliable motionvectors are widely distributed.

As described, the overall motion information generator 4 of the motiondetecting device 130 of the present embodiment performs statistics shownin FIGS. 22 to 24 based on the motion vector related information S2,thereby generating the overall motion information S4. Thus, it ispossible to generate the overall motion information S4 only using themotion vector related information S2 obtained from the search regionvariable-shaped motion detector 2 without using control information ofthe surveillance camera 5 as in the first preferred embodiment and thelike. This allows simplification of circuit configuration.

FIG. 25 is a block diagram illustrating the configuration of an MPEG-2video encoding device 140 (not including audio functions (the audio I/F17 and the like) according to the present embodiment. Similarly to thesecond preferred embodiment shown in FIG. 7, the search regionvariable-shaped motion detector 7 corresponding to the search regionvariable-shaped motion detector 2 is included in the encoding device 140and overall motion information generating function realized by theoverall motion information generator 4 is provided within the DSP part12. Other components are the same as those in the second preferredembodiment, explanation of which is thus omitted here.

FIG. 26 is a block diagram illustrating the configuration of anotherMPEG-2 video (audio, system) encoding device 150 according to thepresent embodiment. A motion detecting device 131 corresponding to themotion detecting device 130 shown in FIG. 21 is provided within theMPEG-2 video encoding device 150. Other components are the same as thosein the second preferred embodiment, explanation of which is thus omittedhere.

Further, while the present embodiment has described the motion detectingdevice 130 including the search region variable-shaped motion detector 2and overall motion information generator 4, the motion detector 2 itselfhas the effect of obtaining the motion vector mv of high accuracy witheffective processing by receiving the overall motion information S3 fromoutside, similarly to the first preferred embodiment.

Eleventh Preferred Embodiment

FIG. 27 is a block diagram illustrating the internal configuration of asearch region variable-shaped motion detector 6 according to an eleventhpreferred embodiment of the invention. The motion detector 6 correspondsto the search region variable-shaped motion detector 2 of the firstpreferred embodiment shown in FIG. 1.

As shown, the search region variable-shaped motion detector 6 includes asearch region shape control part 23 and a motion detecting part 24. Thecontrol part 23 determines a search region similarly to the searchregion shape control part 21 based on the overall motion information S3obtained from the overall motion information generator 3 and outputs thereference image selection signal S12 and the other selection signal S13for realizing the determined search region, to the motion detecting part24.

Since the search region shape control part 23 indicates selection of thereference image selection signal S12 for only one current image TMB, thecurrent image selection signal S11 becomes unnecessary.

Upon receipt of reference image data SW and current image data TMB, themotion detecting part 24 performs a comparison operation between thereference image data SW and current image data TMB in a search regiondetermined by the reference image selection signal S12, and outputs amotion vector mv based on the result of comparison.

FIGS. 28 to 30 are illustrative examples of relationship of details onselection control of reference image data SW and current image data TMBwith the shape of a resulting search region. In these examples, it isassumed that the number of macroblocks that can be selected by referenceimage data SW is sixteen, operational capability is eight cycles and acomparison between reference image data SW and current image data TMBcan be performed by block in one cycle.

In a first selection example 171 shown in FIG. 28, reference imagesSW11, SW21, SW31, SW41, SW12, SW22, SW32 and SW42 are each subjected toa comparison operation with the current images TMB1, thereby determininga horizontally-oriented rectangular search region 175 having an area of8 macroblocks.

In a second selection example 172 shown in FIG. 29, the reference imagesSW21, SW31, SW22, SW32, SW23, SW33, SW24 and SW34 are each subjected toa comparison operation with the current images TMB1, thereby determininga vertically-oriented rectangular search region 176 having an area of 8macro bocks.

In a third selection example 173 shown in FIG. 30, the reference imagesSW11, SW21, SW22, SW32, SW23, SW33, SW34 and SW44 are each subjected toa comparison operation with the current images TMB1, thereby determininga rectangular search region 177 extending downward to the right havingan area of 8 macro bocks.

As described, in the present embodiment, a selection signal fordetermining a search region outputted from the search region shapecontrol part 23 is only the reference image selection signal S12, inaddition to the effects exerted by the first preferred embodiment.

Twelfth Preferred Embodiment

While the renewal timing (predetermined timing) of search region (searchshape) is on a picture basis in the first preferred embodiment, thesearch shape is determined by macroblock, not by picture in a twelfthpreferred embodiment of the invention.

FIG. 31 is an illustrative example of a screen 190 divided into nineregions. On a screen seen by a driver of a car moving at high speed, amotion is assumed to vary radially from the center of a current imageregion 195. In the present embodiment, a search shape is changed bymacroblock focusing attention to such motion.

Upon receipt of the overall motion information S3, and when theinformation S3 indicates the above-described radial variations ofmotion, the search region shape control part 21 of the presentembodiment selects a search shape by macroblock from among the searchshapes 71 to 75 according to which one of current image regions 191 to199 macroblocks of the current image TMB belong to. Hereinbelow, suchselection will be described.

When macroblocks of the current image TMB belong to any one of thecurrent image regions 192, 194, 195, 196 and 198, the search shape 73 isselected. Besides, for the current image region 192, a direction fromthe reference point of search region is determined such that a variationparticularly in the upward direction is searched as with the searchregion 123 shown in FIG. 20. For the current image region 194, adirection is determined such that a variation particularly in the leftdirection is searched. For the current image region 196, a direction isdetermined such that a variation particularly in the right direction issearched. For the current image region 198, a direction is determinedsuch that a variation particularly in the downward direction is searchedas with the search region 121 shown in FIG. 18.

For the current image region 191, the search shape 71 is adopted.Besides, a direction from the reference point of search region isdetermined such that a variation particularly in the upward direction tothe left is searched. For the current image region 193, the search shape75 is adopted. Besides, a direction is determined such that a variationparticularly in the upward direction to the right is searched.

For the current image region 197, the search shape 75 is adopted.Besides, a direction from the reference point of search region isdetermined such that a variation particularly in the downward directionto the left is searched. For the current image region 199, the searchshape 71 is adopted. Besides, a direction is determined such that avariation particularly in the downward direction to the right issearched.

Since other operations are the same as those in the fourth preferredembodiment, the motion detecting device of the present embodiment exertsthe same effects as those in the fourth preferred embodiment. Further,the motion detecting device of the present embodiment changes a searchshape and a direction from the reference point of search region bymacroblock. Thus, the motion vector mv of high accuracy can be obtainedwith more effective processing.

Although the present embodiment has described the basic shape of searchregion as parallelogram, other shapes may be adopted.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous modifications andvariations can be devised without departing from the scope of theinvention.

1-16. (canceled)
 17. A search region variable-shaped motion detector fordetecting a motion vector in image data to be encoded, comprising: asearch region shape control part configured to receive overall motioninformation indicative of overall motion of an image and to determine asearch shape as a determined search shape based on said overall motioninformation and to set a search region having an area determined by saiddetermined search shape and predetermined operational capability, and tooutput a selection signal for selecting, by a predetermined unit, atleast one of current image and reference image and reference image insaid image data in accordance with said search region; and a motiondetecting part configured to perform a comparison operation between saidreference image and said current image in said image data based on saidselection signal, thereby detecting said motion vector, wherein saidpredetermined operational capability includes operation capability ofsaid motion detecting part.